Wpm P4.08 Video Object Segmentation Based on Global Motion Estimation/compensation

This paper presents a video object segmentation scheme based on global motion estimation and compensation. For video sequences with or without camera motion, we propose a unified approach to extract the moving object models. This approach combines both the spatial information from current frame and the temporal information derived from global motion compensated f inme. Experimental results demonstrate the validity and performance of our work. INTRODUCTION Video object segmentation is a prerequisite process for object-based coding and content-based retrieval. Spatial-temporal information is extensively employed to video object segmentation [ 1][2]. Temporal segmentation detects the moving areas between consecutive frames, and spatial segmentation detects the edges and boundary information within a frame. For video sequences without global motion, moving areas can be easily detected by thresholding the intensity changes between consecutive frames. The detected moving areas are then combined with the edge information to segment the moving objects. However, for video sequences with global motion, the intensity changes between consecutive frames induce undesired background instead of the foreground moving objects. To solve this problem, in this paper, we propose a unified approach to extract the moving objects from video sequences with and without global motion. This approach first estimates and compensates the background movement. Next, we proceed to temporal segmentation and then combine with the result from spatial segmentation to produce the binary models of moving objects. THE PROPOSED APPROACH The flowchart of the proposed approach is depicted in Fig. 1. We describe the processing steps as follows. 1. Global motion estimation and compensation We adopt an efficient and robust block-based method [3] to estimate the global motion of video sequences in the presence of moving objects. This method consists of the following steps: block motion estimation, calculation of global motion parameters, elimination of the foreground blocks, and iteratively refining the motion parameters until the sets of foreground and background blocks no longer change. Once the precise global motion parameters are obtained using the above-mentioned block-based method [3], the background movement of the curent frame can be accurately compensated. 2. Temporal segmentation After compensating the background movement, the area that is moving differently from the glotlal motion exhibits significant intensity change between two consecutive frames. The change detection mask (CDM) [2] is a binary image that indicates whether the intensity change is larger than a predefined threshold. To separate moving objects from noisy pixels, we apply the connected component-labeling algorithm [4] on CDM and then extract the connected components with size larger than a threshold. 3. Spatial segmentation Spatial segmentation detects the edges and boundaries of the objects. Since Canny operator [ 5 ] is an effective edge detector, we combine the edges pixels (detected by Canny operator and the pixels belonging to the connected components to produce the binary model of the moving objects. EXPERIMENTAL RESULTS Fig. 2 shows the motion models extracied from sequence “Hall monitor”, and Figure 3 shows the binary models of multiple moving objects extracted from sequence “Trevor”. Fig. 4 and 5 demonstrate that our proposed approach extracts the moving objects successfully for video sequences with either slow (e.g., “Coastguard”) or fast camera motion (e.g., “Stefan”). ACKNOWLEDGEMENT This work was partially supported bj, the Republic of China National Science Council under contract no. NSC 89-22 13-E-07-044. REFERENCES M. Kim, J. G. Choi, D. Kim, H. Lee, M. H. Lee, C. Ahn, and Y. S. Ho, “A VOP generatton tool: automatic segmentation of moving objects in image sequences based on spatio-temporal information,” IEEE Trans. Circuits Syst. Video Techol., vol. 9, no 8, pp. 1216-1226, Dec. 1999. T. Meier and K. N. Ngan, “Video segmentation for content-based boding,” IEEE Trans. Circuits Syst. Video Techol., vol. 9, no 8, pp. 1190.12.3, Dec. 1999. 168 * Correspondence: Email: [email protected] 0-7803-6622-0/01 $10.00